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ABSTRACT 



The latent structure of the reading test of the 
International Association for the Evaluation of Educational Achievement (lEA) 
Reading Literacy study was investigated. The use of latent class modeling for 
investigating the measurement properties of a large-scale reading assessment 
database is demonstrated. The study focused on response data from the United 
States, Canada, Hong Kong, and Denmark, for a total sample of 15,304 
nine-year-olds. One of the main research questions was whether the type of 
text (narrative, expository, and documents) elicits different types of 
reading comprehension. Findings indicate that there is no empirical evidence 
of three separate text types for any of the four countries, but there is 
empirical evidence of two separate reading domains . One latent variable 
represents a general reading domain associated with all three text types, and 
the other represents a prose reading domain associated only with expository 
and narrative text types. The latent class modeling approach provides a way 
to examine the latent structure of the lEA reading test that avoids many 
problems that may occur when applying factor-analytic methodology. (Contains 
3 figures, 12 tables, and 30 references.) (SLD) 
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Purpose 

The purpose of this study was to investigate the latent structure of the reading test of the 
lEA Reading Literacy Study (Elley, 1992) and to demonstrate the use of latent class modeling 
for investigating the measurement properties of a large scale reading assessment database. The 
psychometric properties of reading are a hotly debated issue within the reading research 
community. Should reading comprehension be conceptualized as a unitary or a domain 
specific ability? This study demonstrates a new psychometric approach to establishing the 
domain structure of the lEA reading test using Latent Class Analysis (LCA). The study 
consists of two phases. In Phase I, the main research question addressed was whether three 
distinctive domains of reading text types can be identified and whether they can be 
distinguished from each other. In Phase II, the homogeneity of the latent structure identified in 
Phase I was examined across countries. 

Theoretical framework 

The International Association for the Evaluation of Educational Achievement (lEA) 
Reading Literacy study is summarized in Elley (1992). About 30 countries participated in the 
study in which reading achievement data were collected. The data collection was performed in 
1990-1991 from representative samples of two age groups: nine-year-old and 14-year-old. The 
present analyses focused on test results from the nine-year-old cohort in four selected countries: 
Canada, Denmark, Hong Kong, and United States. One basis of test construction stemming 
from the reading research literature was that reading comprehension should be diversified 
enough to include three text types: Narrative, Expository, and Documents (Elley, 1992). Thus, 
the tests used were a composition of different material each representing one of these three 
domains. Reading comprehension is postulated, to certain degree, to be dependent on the 
reading domain. Such a division of reading domains, however, is regarded as more pragmatic 
than conceptual (Weaver & ^ontsch, 1991) and sometimes has provoked controversy. Some 
researchers find that it is impossible to draw absolute boundaries between narrative and 
expository texts due to similarities of the cognitive processing involved in performing each 
tasks (Spiro & Taylor 1980); others argued that document search and prose reading involved 
relatively distinctive cognitive processes (Guthrie & Kirsh, 1987). 

A Rasch scaling (Rasch, 1960) performed separately for each domain provided some 
justification that items from each of the three domains reflects a single construct (Elley, 1992). 
It is an empirical question whether such an assumption is indeed valid. A few studies have 
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suggested that there is no evidence for three separate reading comprehension factors (Balke, 
1995; Atash 1996). 

Distinction Among Three Text Types 

An important implication for reading assessment as a result of the separation of text 
type is that it becomes very difficult to compare a student's performance across domains. 
Literature suggests that many children have difficulty understanding expository materials after 
spending most of their study time with narratives (Spiro & Taylor, 1980) (Berkowitz & Taylor, 
1981). Some researchers suggest that students perceive expository materials to be more 
difficult than narratives (Alvermann & Boothby, 1982). Others found that students are less able 
to recall expository text (Zabrucky & Ratner, 1992). It had been suggested that the text-specific 
hierarchical structure of expository text is difficult for students to discern and retain. Another 
often cited reason is that the vocabulary or content of expository material is unfamiliar to the 
children. However, there is very little empirical evidence to support this claim. Spiro et al. 
[1980] conclude that most studies comparing the text difficulty result in ambiguous 
interpretations. 

Researchers in adult literacy have argued that the cognitive processes involved in 
locating information in text do not appear to be identical to those involved in reading 
comprehension (Guthrie & Kirsch, 1987; Guthrie & Mosenthal, 1987). They contented that 
since reading comprehension had been generally defined as recalling of text, a separate 
operational definition is needed for tasks such as searching information in documents or 
locating a particular number in the tables. Since the processing requirement for these two types 
of tasks is different, competencies of reading comprehension and locating information would be 
relatively independent. It was concluded that reading tasks that required imderstanding and 
remembering lengthy sections of text were independent, statistically, fi-om tasks that required 
locating specific details in mariuals, schematics, and periodicals (Guthrie & Kirsch, 1987). 

Hierarchical Models for Reading Comprehension 

Carroll (1993) identified some primary factors in the language domain and many of the 
primary factors are associated with the reading tests. In a re-analysis of 148 datasets, a Verbal 
or Printed Language Comprehension factor measured by vocabulary, information and reading 
comprehension tests was identified. Carroll concluded that there is evidence for a unitary 
language ability, even though some specific subskills were identified in different area of verbal 
domain. It was also demonstrated that the primary factors in the verbal domain are dominated 
by the second-order ‘G’ factor, so there is sonie evidence that general verbal ability is a central 
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component of crystallized intelligence. Carroll also found evidence for special reading 
comprehension factors separate from other factors in the verbal domain in a re-analysis of 10 
datasets. However, he cautioned that factor loaded with reading comprehension tests often 
have high loadings on various kinds of reasoning tests. Therefore, the common element in this 
factor is probably not reading comprehension per se, but reasoning ability. 

Balke (1995) investigated the domain structure of the lEA reading literacy test for five 
Nordic countries on both the 9-and 1 1 -year old populations. The main research question was 
whether three different reading factors (narrative, expository, and document) could be identified 
from the data. The analysis suggested that three text type factors could not be differentiated 
because items from expository and narrative texts form one factor that is clearly distinct from 
the document items. Several attempts to separate narrative and expository text types had 
proven to be impossible. The model was not identified once such restrictions were imposed on 
the model. The final analysis shows that there are two structural equation models that best 
explain the variations in the data. One model (Model 1) contains a general ability factor, the 
passage factor, and expository-narrative text type factor; the other model (Model 2) contains a 
general ability factor, the passage factor, and document text type factor. It was concluded that 
both models were equally representative of the data because the differences in fit statistics are 
ignorable. However, Model 2 is chosen as the preferred model because it shows more 
consistencies across countries. 

There are some drawbacks in the analytical approaches presented in Balke’s study. 
First, the product moment correlations instead of the tetrachoric correlations were used. 
Analysis based on the product moment correlations of dichotomous variables may distort the 
correlations and hence to the factor-analytic results derived from them (Comrey, 1978). 
Furthermore, neither of the models provided acceptable fit to the data according to the chi- 
square statistics. Therefore, there is a lack of statistical and empirical justifications for choosing 
either model as the preferred model. 

Past researches in examining the internal structure of reading tests rely on factor- 
analytic models (Carroll, 1993; Balke, 1993; Atash, 1996). Most of these work are predicated 
on the assumption that the data followed multivariate normality: an assumption that is left 
unchecked and often invalid. In contrast, latent class models do not impose distributional 
assumption on the data. 
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Method 

The basic premise of latent class analysis (LCA) is that the observed dependency 
among manifest variable is due to each variable's relationship to the latent variables (Lazarsfeld 
& Henry, 1968; McCutchen, 1987). If a given latent class model holds, then one can say that 
the relationship among the manifest variables is explained by the latent variables (Aitken, 
Anderson, & Hinde, 1981; Goodman, 1 974). The basic approach of LCA is to identify a set of 
mutually exclusive latent classes based on the observed item responses. In this study, a series 
of latent class models corresponding to different structural relationship among the domains 
were formulated and tested starting at the passage level and then proceeds by aggregating 
variables across passages. This strategy makes it possible to identify general and specific 
components of reading domains. 

Data 

The current study focused on response data from four countries: U.S.A., Canada, Hong 
Kong, and Denmark. The selection of these countries provides a broad range of variations in 
terms of the distributions of test scores across reading domains. The sample size is 2,565 for 
Canada, 3,327 for Denmark, 3,186 for Hong Kong, and 6,390 for the United States, resulting a 
total sample size of 1 5,304. The analyses in this study were conducted using LCAG, version 
2. 1 Oa (Hagenaars & Luijkx, 1 987) and Lem, version 0. 1 1 (V ermunt, 1 993). 

Fifteen reading passages were administered to the nine-year-old population in the lEA 
study. Each passage was followed by three to six questions. The items were given in multiple- 
choice format except for the four items in the passage "Buses,", where the examinees had to 
write the answers consisting of a number or a name. Due to the large number of non-response 
or non-reached items for sorqe countries, response data for some passages are not reliable. 
Nine passages were selected in this study: three document passages (“Buses”, “Maria”, and 
“Temperature”), three expository passages (“Marmot”, “Quick Sand”, and “Walrus”), and three 
narrative passages (“Bird”, “No Dog”, and “Shark”), with a total of 41 items. 

Since an efficient estimation method for a large number of items for restricted Latent 
Class Models (LCM) is not yet available, passage scores created at an earlier study (Yen, 1997) 
were used to assess the domain structure. The latent structure for each passage within each 
country was first identified, the parameter estimates were inspected to identify a dominating 
mastery class in the population. Examinees belonging to the mastery class were assigned a 
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passage score of ‘ 1 ’ and those belonging to other classes were assigned a passage score of ‘O’. 
Across nine passages, there are (2^)=512 response vectors consisting of the classification 
decisions for each respondents. 



Phase I: Domain Structure Within Each Country 

The first research question examined was the domain structure across the nine passages 
within each country. The basic modeling framework is the domain structure analysis based on 
log-linear path model with latent variables proposed by Goodman (1974), Hagenaars, (1990) 
and Vermunt (1993). The log-linear path model may be conceptualized as consisting of two 
parts: a measurement part and a structural part. The structural part of a log-linear path model 
specifies the relationship between the latent variables. The measurement part of the model 
specifies the relationship between the latent construct and their indicators. Latent class models 
can be considered as the measurement part of a log-linear path model where the relationship 
between the latent construct and their indicators is specified. 

A basic latent class model may be represented in terms of Lazarsfelds's original 
parameterization (Goodman, 1973) or in terms of log-linear modeling (Haberman, 1979). 
Starting with the former and following Goodman's notation, the basic equation of a latent class 
model with three manifest variables A through C with indices a, b , and c and one latent 
variable X with index x may be stated as follows: 



^.rahc = ( 1 ) 

Where m denotes the expected frequency, N denotes the total sample size. Latent class 
proportion is denoted as % and the conditional probability of being in level a of latent class x is 
denoted as 7ta|x- All parameters in equation (5) are probabilities and subject to the standard 
restrictions. Secondly, the same latent class model can be expressed in terms of a log-linear 
model: 



( 2 ) 



In Equation (2), m^abc denotes the expected frequencies of marginal table XABC, 
7 / refers to the fixed margin and the X's are ordinary log-linear parameters. All parameters in 

equation (2) are subject to the standard restrictions of log-linear models. From the log-linear 
path model perspective. Equation (2) specifies a logit model with A, B, and C as independent 
variables andXas dependent variable. 
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It can be verified that both parameterizations of the latent class model implement the 
assumption of local independence. Haberman (1979) also demonstrated that the relation 
between the conditional probabilities of equation (1) and the X parameters of equation (2) is 



^u\x ~ 



exp(A;! + A^) 

X„exp(A>A^) 



( 3 ) 



Conversely, it can be verified that the log-linear parameters in equation (3) can be 
expressed in terms of the conditional probabilities in equation (2) with some algebraic 
manipulations. For the rest of this paper, the measurement part of each model was presented 
using Goodman’s parameterization in order to demonstrate the constraints imposed on the 
model. The structural part of the model in this paper was specified as a long-linear model in 
order to demonstrate the structural relationship among the latent variables. 



Latent Class Models in investigating Domain Structure 

To investigate the latent structure of three reading domains, seven types of latent class 
models were tested for adequate absolute fit and relative fit to the data : 

Unconstrained one latent-variable model (Model 1): This model is equivalent to a latent class 
model that assumes one latent variable with three levels or latent classes. The measurement 
part of the model can be obtained by expanding the number of manifest variable to nine in 
Equation ( 1 ) and let x=3 . 

Three-independent domain Model (Model 2): This model assumes that the relationship among 
the manifest variables can be explained by introducing three latent variables, each 
representing a text domain. Each latent variable has three classes. It further specifies that 
the latent variables are independent of one another. Thus, the model states that each text 
domain explains the relationships among the passages underlying the domain and three 
reading domains are independent of one another. Model 2 specifies that Y, Z, and U are 
three latent variable representing document, expository, and narrative domain, respectively. 
Manifest variables A tlirough C are assumed to be explained by latent variable Y, D through 
F explained by Z, and G through / explained by U. Furthermore, there is no direct 
relationships among X, T, and Z. The measurement part of Model 2 can be stated as: 

^ yzuahcdefghi ~ ^ ^ u\y^ h\y^ i.\y^ d\z^ f\z^ h\u^ i\u ( 3 ) 

The structural part Model 2 may be stated as: 
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( 6 ) 



Three-related reading domain Model (Model 3): In contrast to Model 2, this model relaxes the 
assumption that three latent variables are independent of one another. Therefore, the 
measurement part of the model is the same as specified in equation (5). However, the 
structural part of the model now includes two-way and three-way interactions among Y, Z 
and!/: 



log = iZ' + + < + k: + 



'iZV 



^YZIJ 



(7) 



For the following models, the restrictions imposed on the models can be derived by 
following the algoritlim presented above. Details of the derivation are not presented here. 

One general reading domain and one specific text domain (Model 4-6): This model postulates 
one latent variable has direct effect on all nine passages and the other has direct effect only 
on the reading passages underlying a specific text type. The general reading domain has 
three classes and the text domain has two classes. Furthermore, the latent variables are 
independent of each other. For example. Model 4 (Table 1) assumes that in addition to a 
general reading domain, document texts could be identified as a separate reading domain. 
This also means that the general reading domain is more influenced by the expository and 
narrative passages. 

One general reading domain and a composite of two text types (Model 7-9): In contrast to the 
previous model, this model assumes that the text domain is a composite of two text types. 
The general reading domain has three classes and the composite text domain has two 
classes. For example. Model 7 (Table 1) assumes that there exist two independent latent 
variables: one being the general reading domain and the other being the combination of 
document and expository text type. That is, the general reading domain is more influenced 
by the narrative texts. 

One general reading domain, a composite of two text types, and a specific text domain (Model 
10-12): In contrast to the previous model, this model includes an additional text type as a 
separate latent variable with two classes. For example. Model 10 (Table 1) assumes that 
there exist one general reading domain, one document-expository text domain, and one 
narrative text domain. 



One general reading domain and three independent text domains (Model 13): In contrast to the 
three-independent text domain model (Model 2), this model postulated an additional 
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reading domain that has direct effect on all reading passages. This model is equivalent to a 
bi-factor model. It is the most complex structure examined in this phase. 



Model No, 


Descriptions 




1 


One Reading Domain 




2 


Three-Independent Text Domains 






Three-Related Text Domains 




4 


General and Document Domain 




5 


General and Expository Domain 




6 


General and Narrative Domain 




7 


General and Composite of Document, Narrative 




8 


General and Composite of Document, Expository 




9 


General and Composite of Expository, Narrative 




10 


General, Composite of Nar. and Exp., and Document 




II 


General. Composite of Nar. and Doc., and Expository 




12 


General. Composite of Exp. and Doc., and Narrative 




13 


General. Document, Expository, and Narrative 





Table 1 ; Descriptions of Domain Structure Models in Phase I 



For each country, a total of 13 latent class models were compared to determine the 
model that best represents the domain structure of that country. Table 1 presents the 
descriptions of all the models being compared for this phase. For model 1, 2, and 3, each latent 
variable has three levels. For the rest of the models, the General reading domain has three 
classes whereas the text specific reading domain has two classes. Models with different 
number of levels for the latent classes were considered in an earlier study (Yen, 1997). The 
absolute fit of each hypothesized model was examined using the likelihood ratio chi-square 
statistic. Competing models that provided acceptable fit were compared with respect to their 
parsimony using AIC criterion (Akaike, 1973) and model with minimum AIC was chosen as 
the preferred model to represent the domain structure for that country. 

Phase II: Domain Structure Across Countries 

The purpose of this phase was to examine whether the domain structure identified in 
Phase I was homogeneous across the four nations. Under the fiamework of the multiple-group 
LCAs (Clogg & Goodman, 1985), one may test the hypothesis with respect to a heterogeneous 
or a homogeneous latent stmcture across groups. Heterogeneous latent structure applies when 
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there is no across-group restrictions imposed on the parameters. When across-group equality 
restrictions are imposed on the parameters, then the model is said to have a homogeneous latent 
stmcture across groups. To test the hypothesis that a given latent structure is adequate in 
explaining the relationship among the manifest variables across groups is equivalent to testing 
whether the groups are homogeneous with respect to the item structure. This can be achieved 
by imposing equality restrictions on the latent proportions and/or the conditional probabilities 
across groups (Clogg & Goodman, 1985). 

Results 

Domain Stmcture Within Country 

Likelihood ratio chi-square statistics (G^), degrees of freedom, and AIC information 
criterion (Akaike, 1973) for each model are reported in Tables 2 to 5. Model 9 had the smallest 
AIC among all 13 models for all four countries. This model provides support for existing 
theory about the domain structure of the lEA reading test in that a two-independent latent 
variable stmcture is necessary to explain the relationship among the manifest variables. One 
latent variable represents a general reading ability \vith three classes; the other latent variable 
represents expository/narrative reading with two classes. 



Model 




df 


AIC 


1 


390.719 


482 


-573.281 


2 


1460.550* 


478 


504.550 


3 


363.515 


458 


-552.485 


4 


375.726 


475 


-574.274 


5 


389.141 


475 


-560.859 


6 


371.365 


475 


-578.635 


7 


366.775 


469 


-571.225 


8 


362.184 


469 


-575.816 


9 


357.269 


469 


-580.731** 


10 


354.172 


462 


-569.828 


II 


361.710 


462 


-562.290 


12 


364.026 


462 


-559.843 


13 


360.397 


461 


-561.603 



p < 0.05. **: Minimum AIC 



Table 2: Model Comparisons of Domain Stmcture for Canada 





Mode! 




df 


AlC 


1 


416.999 


482 


-547.001 


2 


1823.202* 


478 


867.202 


3 


397.620 


458 


-518.380 


4 


400.226 


475 


-549.774 


5 


413.645 


475 


-536.355 


6 


413.547 


475 


-536.453 


7 


394.836 


469 


-543.164 


8 


395.937 


469 


r -542.063 


9 


379.249 


469 


-558.751** 


10 


374.325 


462 


-549.675 


11 


389.731 


462 


-534.269 


12 


389.006 


462 


-534.994 


13 


393.923 


461 


-528.077 



*:p < 0.05. **; Minimum AlC. 



Table 3; Model Comparisons of Domain StrucUire for Denmark 



Model 




df 


AlC 


1 


423.533 


482 


-540.467 


2 


1837.462* 


478 


881.462 




389.810 


458 


-526.190 


4 


387.969 


475 


-562.031 


5 


421.630 


475 


-528.370 


6 


416.630 


475 


-533.370 


7 


383.052 


475 


-554.948 


8 


380.809 


475 


-569.191 


9 ; 


367.455 


475 


-570.545** 


10 


362.700 


462 


-561.300 


11 


380.680 


462 


-543.322 


12 


379.592 


462 


-544.408 


13 


379.891 


461 


-542.109 



p < 0.05. *•; Minimum AlC. 



Table 4: Model Comparisons of Domain StrucUire for Hong Kong 
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Model 




df 


AIC 


1 


484.243 


482 


-479.757 


2 


4265.945* 


478 


3309.945 


3 


428.363 


458 


-487.637 


4 


448.614 


475 


-501.139 


5 


472.570 


475 


-477.430 


6 


451.154 


475 


-498.846 


7 


441.104 


469 


-496.896 


8 


409.935 


469 


-528.065 


9 


405.709 


469 


-532.291** 


10 


401.352 


462 


-522.647 


11 


429.538 


462 


-494.462 


12 


403.070 


462 


-520.930 


13 


418.821 


461 


-503.179 



*: p < 0.05. **: Minimum AIC. 

Table 5: Model Comparisons of Domain Structure for U.S.A. 

Estimates of latent class parameters for each country are presented in a two by three 
contingency table with six cells in Tables 6 to 9. The row margins and the column margins 
contain the latent class proportions for the Exp/Nar reading domain and the General reading 
domain, respectively. It is apparent from the row and column margins that the two latent 
domains are independent. Overall, there is a relatively even split between the row margins 
whereas more variation is noted among the column margins. Wide variations were found for 
the General reading domain where the majorities of the respondents belonging to the 
intermediate class and very few were nonmasters. The distribution of latent class proportions 
for the Exp/Nar domain was fairly evenly split between two classes. Canada and Denmark have 
slightly more masters than nonmasters while the opposite is true for Hong Kong and U.S.A. 
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Exp/Nar Domain 


General Reading Domain 






Nonmaster 


Intermediate 


Master 


Total 




Nonmaster 


0.078 


0.233 


0.130 


.441 




Master 


0.099 


0.295 


0.165 


.559 




Total 


0.177 


0.528 


0.295 


1 



Table 6; Latent Class Parameters for Canada 



Exp/Nar Domain 


General Reading Domain 




Nonmaster 


Intermediate 


Master 


Total 




Nonmaster 


0.050 


0.194 


0.182 


.426 




Master 


0.067 


0.261 


0.245 


.573 




Total 


0.117 


0.455 


0.427 


1 



Table 7: Latent Class Parameters for Denmark 



Exp/Nar Domain 


General Reading Domain 






Nonmaster 


Intermediate 


Master 


Total 




Nonmaster 


0.066 


0.237 


0.245 


.548 




Master 


0.055 


0.195 


0202 


.452 


Total 


0.121 


0.432 


0.447 


1 




Table 8: Latent Class Parameters for Hong Kong 
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Exp/Nar Domain 


General Reading Domain 




Nonmaster 


Intermediate 


Master 


Total 




Nonmaster 


0.100 


0.266 


0.171 


0.537 




Master 


0.086 


0.230 


0.147 


0.463 




Total 


0.186 


0.496 


0.318 


1 



Table 9: Latent Class Parameters for U.S.A 



Model 


General 


Nar/Exp 




df 


AIC 


Hi 


9 free, a free 


9 free, a free 


1509.682 


1876 


-2142.318 


H2 


9 free, a free 


9 free, a equal 


1535366 


1912 


-2288.634** 


H3 


9 free, a free 


0 equal, a equal 


1546.061 


1915 


-2283.939 


H4 


9 free, a equal 


9 free, a free 


2112.475 


1957 


-1801.525 


Hs 


0 equal, a equal 


9 free, a free 


2775.203 


1963 


-1250.797 


He 


9 free, a equal 


0 equal, a equal 


6178.972* 


1996 


2286.972 


H7 


0 equal, p equal 


9 free, a equal 


5715.697* 


1999 


1717.697 


Ha 


9 free, a equal 


9 equal, a equal 


4813.249* 


1993 


817.249 


Hg 


0 equal, a equal 


9 equal, a equal 


8971.427* 


2001 


4969.427 



1 : 0 denotes latent class proportion 
2: a denotes conditional probability 
*: p<0.05. **: Minimum /4/C. 



Table 10: Cross-country Latent Stmcture Analysis 



Domain Structure Across Couiitries 

Tlie research question examined at Phrase II was vdiether the domain stmcture is the 
same across four countries. Nine simultaneous latent stmcture models were tested using 
country as the grouping variable. For ease of discussion, these models are grouped into three 
major categories: 

1 . Heterogeneous Model (H Q: This model tested the assumption that the latent stmcture based 
on Model 9 holds simultaneous across countries, however, conditional probabilities and 
latent class proportions are unconstrained. 
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2. Homogeneous Models (H s and H 9): Model Hg specified that conditional probabilities are 
the same for the four countries. Under the fimnework of simultaneous latent structure 
analysis, if conditional probabilities were the same across groups, then latent structure is 
considered to be homogeneous. By comparing Models H| and Hg, the assumption of 
homogeneity across group is tested. Since they are nested models, comparisons between 
them may be made using the likelihood ratio chi-square difference test (Hagenaars, 1990). 
Model Hg is the more restrictive model than Model Hg in which latent class proportions are 
constrained to be equal across countries. 

3. Partial Homogeneous Models (Models H 2 to H?): Models H 2 to H? are a series of models in 
which the latent Structure for one reading domain is homogeneous but the latent structure 
for the other domain is heterogeneous across groups. 





Nonmastery 


Intermediate 


Mastery 


Total 


Canada 


0.086 


0.039 


0.039 


0.163 


Denmark 


0.067 


0.018 


0.069 


0.154 


Hong Kong 


0.019 


0.091 


0.099 


0.209 


U.S.A. 


0.073 


0.229 


0.171 


0.473 


Total 


0.245 


0.377 


0.378 


1.000 



Table 11. Latent Class Parameters Across Country: General Reading 





Nonmastery 


Mastery 


Total 


Canada 


0.122 


0.042 


0.164 


Denmark 


0.064 


0.091 


0.155 


Hong Kong 


0.109 


0.100 


0.209 


U.S.A. 


0.216 


0.257 


0.473 


Total 


0.511 


0.490 


1.000 



Table 12. Latent Class Parameters Across Country: Nar/Exp Reading 

As shown in Table 10, the heterogeneous model. Hi, fits the data well. When cross- 
country restrictions were imposed on the conditional probabilities for both latent domains 
(Model Hg), the fit to the data deteriorated drastically. Three of the six partial homogeneous 
models fit the data. Among the five models that fit the data. Model H 2 yielded the smallest 
AIC and therefore, was chosen as the preferred model. Model H 2 specifies a heterogeneous 
structure for the General reading domain and a homogeneous structure for the Exp/Nar reading 
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domain across country. In other words, the conditional probabilities for the Exp/Nar reading 
may be constrained to be equal across countries, while the conditional probabilities associated 
with the General reading can not be constrained. Furthermore, the latent class proportions for 
both reading domains are different across countries. Line graphs of average conditional 
probability of correct response are represented by country in Figure 1 to Figure 3 for each latent 
class. There were relatively small country differences for the mastery class while larger country 
differences were found in the nonmastery class. Overall, nonmasters in Hong Kong have a 
distinctly higher probabilities of getting most of the passages ("Maria," "Temperature," 
"Marmot," and "Shark") correct than other classes. In contrast, nonmasters in Canada have 
lower probability of getting the passages correct than other countries. Latent class parameter 
estimates by country' are presented in Tables 1 1 and 12 for each reading domain. 
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Figure 1 : Average Conditional Probability Across Country-Mastery Class 
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Figure 2: Average Conditional Probability Across Country-Intermediate Class 
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Figure 3: Average Conditional Probability Across Country-Nonmastery Class 



Discussion 



One of the main research questions examined in the study was whether the type of text 
elicits different type of reading comprehension. The results indicate that there is no empirical 
evidence of three separate text types for any of the four countries examined. However, there is 
empirical evidence of two separate reading domains. One latent variable represents a General 
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reading domain associated with all three text types, the other represents a Prose reading domain 
associated only with the Expository and Narrative text types. Passages with documentary 
information do not form a specific text domain. In other words, not three, but only two reading 
types were identified. 

The General reading domain is to be regarded as a complex latent variable. 
Psychometrically, it accounts for the general cognitive differences among individuals in 
responding to all reading passages examined. At the most general level, this latent variable 
may include variations depending on test format, test administration, and an individual's test 
proficiency. Specifically, this latent variable is associated with some analytical ability to deal 
with reasoning, making inferences and contrasts, and integrating or synthesizing the 
information in the content of reading passages in general. Since items fi-om the passages with 
documentary information do not form a specific text domain, the General reading domain is to 
be regarded as highly influenced by individual differences in locating information in texts 
where general intellectual ability is emphasized. Thus, the General reading domain represents 
the set of cognitive skills common to the respondents in dealing with the reading passages in 
general. This set of skills consists of both the verbal component and the quantitative 
component of the reading skills the individuals possess. 

In contrast to the General reading domain, the Prose reading domain seems to reflect 
specific linguistic abilities that are associated with recalling and understanding written prose. 
This set of skills reflect the individual differences in word meaning or vocabulary, language 
processing, and world knowledge held in verbal form. Since this latent variable would be fi:eed 
fi'om the variations fi-om the general cognitive differences among individuals. It may be 
regarded as more reflective of the individual differences in reading comprehension. In the 
context of this model, reading comprehension of written prose is therefore regarded as a rather 
specific ability. 

This latent structure helps to explicate the relationship between Document and 
Narrative-Expository text type. The document items require reading some amount of 
continuous text, but the major source of the difficulty seems to come fi-om locating information 
in the tables, graphs, and maps, and processing the information in the passages. Therefore, 
poor reading skills certainly would affect the performances, but good reading skills are not 
sufficient in explaining the successful performances on these passages. Instead, the analytic 
abilities to deal with reasoning and processing information are more important in explaining the 
performances in these items. Therefore, Document items did not form a distinct text type once 
the individual differences of performance on these items has been accounted for by the General 
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reading domain. These interpretations by no means suggest that the Document items can be 
interpreted as a pure measure of reasoning ability. Instead, Document items to some extent 
capture the same type of reading ability as is measured by the Narrative and Expository items. 

Many of the Expository and Narrative passages also require inferences and complex 
reasoning. However, once the individual differences in making inferences and logic reasoning 
had been accounted for by the General reading domain, a separate Prose reading domain 
remains in the model. Therefore, there is evidence of a separate Prose reading domain that is 
distinct fiom the General reading domain. Since the major difference between the Narrative- 
Expository passages and Document passages is the amount of continuous text to process, the 
Prose reading domain could be regarded as a text-processing domain that reflects certain 
linguistic ability. 

In essence, the results show that there is some evidence for a general reading domain, 
and the Expository and Narrative text types measure the same text processing skills. These 
results conform to the finding from previous factor analytic studies by Carroll in 1993. The 
findings from the current study can also be compared with the analysis conducted by Balke 
(1996) in which the domain structure of the lEA reading test was analyzed using structural 
equation modeling. While the domain structure identified in this study bears some similarities 
to the models proposed by Balke (1996), there are some fundamental differences in the 
methodology. The significant chi-square statistics in Balke's study indicate that neither of the 
proposed models fit the response data for any of the five countries. Furthermore, the Goodness 
of Fit Statistics were far from ideal for both models for any countries. Although these problems 
may be reflective of the limitations of the fitting statistics, the interpretability of the models was 
still questionable. In contrast, the domain structure identified in this study fit the response data 
for all four countries and offer unambiguous interpretations about the nature of the relationship 
among the reading domains. 

Tire cross-country similarity in the domain structure provided some evidence for the 
stability of this domain structure. Since the same domain structure hold simultaneously across 
four countries, it can be concluded that the domain structure is consistent across countries. The 
simultaneous latent structure analysis across coimtries provided some further insight with 
regard to the nature of the latent variables. The cross-country heterogeneity in the conditional 
probabilities associated with the General reading domain indicates that the individuals in four 
countries differ in their general reading skills. However, the conditional probabilities may be 
constrained to be equal across countries in terms of the Prose reading domain. Therefore, the 
item difficulties do not seem to differ across country with respect to the Prose reading domain. 
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Conclusions 

The internal structure of tests is a critical issue of both classical and modem test theory. 
Factor-analytic methodology continues to be an active area of research. Most of this work is 
predicated on the assumption that examinee ability can be represented by one or mote 
continuous latent variables. Under the framework of latent class models, examinees are 
assumed not to conform to a continuous latent attribute, but to exactly one of some small 
number of discrete latent classes. Unlike the factor-analytic method and latent trait models, 
latent class models involves minimum assumption about the data. The only assumption in the 
formulation of the latent class model is the assumption of local independence. In fact, what all 
these different methods have in common is their reliance on the axiom of local independence 
(Mooijaart, 1982). For factor-analytic and latent trait analysis, however, models are derived 
fr’om this axiom by making further assumptions about the data. 

Tire latent class modeling approach provides a way to examine the latent structure of 
the lEA reading test that avoids many problems that may occur when applying factor-analytic 
methodology. Current practice in structural modeling is limited to representing the covariance 
structure when the first and second moments are assumed and to the distribution theory based 
on multivariate normality. However, the assumption of marginal multivariate nor mali ty is 
generally ignored in a factor-analytic approach. Several studies have shown that statistics based 
on normal theory can be seriously in error when the distribution is in fact non-normal (Bentler, 
1983) (Browne, 1984). In particular, simulation studies showed that excessive kurtosis usually 
eliminates asymptotic efficiency and makes that estimated asymptotic covariance matrix and 
chi-square estimators incorrect (Browne, 1984). Therefore, the tests of fit of a model and 
standard errors for parameter estimates derived under the assumption of multivariate normality, 
such as those obtained by LISREL (Joreskog & Sorbom, 1988) should be not employed if the 
distribution is not normal (Browne, 1984). 

Balke's study (1995) analyzed the dimensionality of the lEA reading test using the 
correlation matrices generated firom dichotomous item responses. Severe distortion can be 
introduced into the correlation matrix which either imposes artificial limits on the size of the 
correlation or inflate correlation values as compared with what probably would have occurred 
with continuously measured, normally distributed variables. Furthermore, the assumption of 
multivariate normality is usually violated when analyzing dichotomous response variables as if 
they were continuous response variables (Bollen, 1989) (Comrey, 1978). Researchers 
demonstrated that the covariance structure hypothesis does not hold for categorical indicators 
(Bollen, 1989). Moreover, the distribution of the categorical variables generally differs fi-om 
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that of the continuous indicators. Therefore, the asymptotic covariance matrix is not likely to 
equal those of the continuous indicators. Furthermore, the distribution of the categorical 
variables is likely to be non-normal. Therefore, the parameter estimates and tests of goodness 
of fit may be incorrect. Consequently, the interpretations one may derive from the models may 
be in error. 

Tlie current study demonstrates the usefulness of latent class modeling in addressing 
some measurement issues, including the domain structure and skill compositions of the reading 
tests. Latent class modeling approach provides a way to study the cognitive structure of the 
lEA reading test without making strong assumptions about the data. Under the framework of 
log-linear path models, latent class models can be used to distinguish cognitive skills and skill 
level reflecting item characteristics. Under the framework of multiple group latent structure 
analysis, the nature of the latent parameters can be studied simultaneously across groups. The 
study also demonstrates the usefulness of latent class modeling in large scale, cross-country 
reading assessment. Although reading achievement has been commonly modeled in a 
continuous latent variable framework, this study shows the applicability of latent class models 
to reading achievement. Therefore, it demonstrated that the application of latent class modeling 
is wider than otherwise suggested in the literature. 
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